Confidence level generator for bayesian network

ABSTRACT

A system includes a computer implemented Bayesian diagnostic system. The diagnostic system includes an inferencing engine and a conditional probability table that forms the basis for Bayesian inferences once the diagnostic system is trained. Each inference includes a diagnosis and associated probability of the diagnosis. A confidence generator receives the inferences, and generates a confidence measure for each inference.

BACKGROUND

Bayesian Networks are becoming an increasingly important area forresearch and application in the fields of Artificial Intelligence,Medical Sciences, Bio Informatics and other inference tasks inEngineering. The main reason of popularity is due to the fact thatclassical inferential models do not permit the introduction of priorknowledge into the calculations as easily as in the Bayesian approach.The task of training a Bayesian Network from a training dataset involvestrying to learn a model that best represents the underlying distributionor relationships in the training data. The ultimate goal of the BayesianNetwork is to be able to perform correct inference on new data. Sincethe available training data is never complete in the real world, thetrained model is usually only an approximation of the actual underlyingfunction.

Cross validation is a method commonly used to estimate how accurately atrained Bayesian Network (or any learning model) will perform inpractice. The reasoning behind using a cross validation technique is asfollows: The training of a Bayesian model optimizes the model parametersto make it fit the training data as close as possible. Given anindependent sample of validation data from the same population of thetraining data, it will generally turn out that the model does not fitthe validation data as well as it fits the training data. This isparticularly likely to happen when the size of the training data set issmall, or when the number of parameters in the model is large. Crossvalidation is a way to estimate how good the model will perform withfuture data.

SUMMARY

A system includes a computer implemented Bayesian diagnostic system. Thediagnostic system includes an inferencing engine and a conditionalprobability table that forms the basis for Bayesian inference once thediagnostic system is trained. Each inference includes a diagnosis andassociated probability of the diagnosis. A confidence generator receivesthe inferences, and generates a confidence measure for each inference.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a high level flowchart of a computer implemented method ofdetermining a confidence level for a Bayesian network according to anexample embodiment.

FIG. 1B is a high level block diagram illustrating a Bayesian networkdiagnostic system having a confidence generator according to an exampleembodiment.

FIG. 2 is a flow diagram illustrating further detail of the methodillustrated in FIG. 1 according to an example embodiment.

FIG. 3 is a conditional probability table on which inferences are basedaccording to an example embodiment.

FIG. 4 is a graph illustrating a z-curve in which a shaded arearepresents a confidence level according to an example embodiment.

FIG. 5 is a table illustrating various confidence levels according to anexample embodiment.

FIG. 6 is a block diagram of an example computer system for implementingthe network.

DETAILED DESCRIPTION

In the following description, reference is made to the accompanyingdrawings that form a part hereof, and in which is shown by way ofillustration specific embodiments which may be practiced. Theseembodiments are described in sufficient detail to enable those skilledin the art to practice the invention, and it is to be understood thatother embodiments may be utilized and that structural, logical andelectrical changes may be made without departing from the scope of thepresent invention. The following description of example embodiments is,therefore, not to be taken in a limited sense, and the scope of thepresent invention is defined by the appended claims.

The functions or algorithms described herein may be implemented insoftware or a combination of software and human implemented proceduresin one embodiment. The software may consist of computer executableinstructions stored on computer readable media such as memory or othertype of storage devices. Further, such functions correspond to modules,which are software, hardware, firmware or any combination thereof.Multiple functions may be performed in one or more modules as desired,and the embodiments described are merely examples. The software may beexecuted on a digital signal processor, ASIC, microprocessor, or othertype of processor operating on a computer system, such as a personalcomputer, server or other computer system.

A confidence interval is generated for an estimated accuracy of adiagnosis by a Bayesian Network diagnostic system. The confidenceinterval defines how close this estimate is to the actual accuracy ofthe Bayesian Network model. A confidence interval for each (individual)diagnosis made by the Bayesian Network may also be generated.

In one embodiment, the confidence interval of the accuracy of a BayesianNetwork may be calculated at any instance (even during training). ABayesian Network in an intelligent diagnostic reasoner system may nothave sufficient information to make a reasonable diagnosis. Nonetheless,a learning algorithm instrumented to run nightly does continue trainingthe system over time. As the Bayesian Network is trained, its overallexpected accuracy and confidence intervals are expected to improve withtime. Therefore, the overall system accuracy and its confidence intervalmay be used as an indicator of when the diagnostic system is ready forplausible analysis. In one embodiment, the confidence of each individualdiagnosis (or prediction) made by a Bayesian network may be computed.The measure of confidence interval on a per-diagnosis basis allows usersto decide how much importance needs to be placed on a particulardiagnosis.

FIG. 1A is a high level flowchart of a computer implemented method ofdetermining a confidence level for a Bayesian network. At 110, trainingis conducted on a Bayesian diagnostic network. The training may consistof historical input data collected from one or more observations ortests, and the network is provided a corresponding actual diagnosis orresult to be derived from the input data. An at least partially trainednetwork uses the training to determine a probability of a result fromnew input data presented to it. The new input data may correspond tomore training data having known desired results, or actual data.

During training, the network develops a conditional probability tablethat is used for Bayesian inference to provide the probabilities asshown at 120. Results are collected and correlated with actual diagnosesindicated in the training data. Using the collected results, aconfidence level is determined at 130. The confidence level may bedetermined for individual inferences as well as for a cumulativeconfidence for multiple inferences.

FIG. 1B is a high level block diagram illustrating an example Bayesiannetwork diagnostic system 150. System 150 has an inferencing engine 155that utilizes a conditional probability table 160 to generateinferences. The inferences are provided to a confidence generator 165,and an output 170. The confidence generator 165 is also coupled to theoutput 170 to provide a confidence measure of the generated inference.

In one example embodiment as shown in flowchart form in FIG. 2 at 200,an inference engine is applied to detecting failures in components. Thisis just one example of the situations in which a Bayesian inferenceengine may prove valuable in providing probabilities of one or morediagnosis or result. A Line Replaceable Unit (LRU) is a complexcomponent of an airplane, helicopter or ship. LRUs are designed to bereplaced quickly at the organizational level. LRUs speed up repair,because they can be stocked and replaced quickly from inventory,restoring the larger system to service. LRUs also may reduce the cost ofsystems, and increase the quality by spreading development costs of thetype of unit over different models of vehicles.

When faults are detected in LRUs at 215, they are sent to a repair shopfor further analysis and repair. LRUs contain a set of components,including shop replaceable units (SRUs). At the repair shop, techniciansrun a set of diagnostic tests called Test Program Sets (TPS) and make anote of which tests passed or failed at 220. Based on the results of thefailed tests and based on past experience and maintenance logs ofprevious maintenance actions, technicians determine which ShopReplaceable Units (SRU) caused the LRU failure. Diagnosing faults inSRUs takes a long time and are very prone to human errors.

In one example diagnostic system, a Bayesian Network may be used tomitigate the problem. Such a system may shorten the diagnosis time andincrease reliability in Diagnosis by isolating faults quickly andreducing ambiguity. The Bayesian network may be trained with historicaltest and maintenance data as collected at 230 to develop diagnosticmodel enabling intelligent decision support. Once trained at 235, theBayesian network is able to reason intelligently and assignprobabilities corresponding to component(s) that may have gone bad andneed replacement given a set of failed diagnostic tests.

The performance of a diagnostic system may often be dependent on theamount of training data. Field data used as training data is often verysparse or inadequate. There may be many hundreds of tests andcomponents, and often not sufficient data to derive a completeconditional probability table (CPT) for each result. Richness of thedata set (e.g., how well do test failures and bad components correlatein the data, is there noise in data) may also be lacking. The field datausually contains some amount of noise (incorrect data or actual results)due to the fact that part/component numbers after remove and replace aremanually entered in the system.

Although a diagnosis is very valuable, there is no way to tell how“good” is the diagnosis generated by the diagnostic system. In otherwords, there is no way to tell if the inferred probabilities generatedby the system are really meaningful and may be used with confidence.Embodiments described provide confidence measures at 245 for individualdiagnosis 250 generated by a diagnostic system. If a certain diagnosis(e.g., probability of component C_(—)1 being bad) can be backed by ameasure of confidence, then appropriate importance can be assigned tothe diagnosis. In any learning system, when it is just starting tolearn, its performance will obviously be poor and hence diagnosis willnot be correct. A low measure of confidence for individual diagnosiswill indicate that the system is not confident about its own diagnosis.Also, training data is never complete. So, even after the system hastrained sufficiently, if the trained system sees a new input pattern(e.g., a new set of failed tests that it had not seen before), itsdiagnosis of the faulty component will be sufficiently poor. Theconfidence measure would be reflective of such uncertainty.

In one embodiment, field data that diagnostic system uses for trainingis of the form shown in a conditional probability table (CPT) 300 inFIG. 3. The data represents how many times a test was observed to havefailed when the corresponding component was bad. For example, at 310,the 1^(st) row implies that there were five observations where test T1failed when B1 was the bad component (had to be replaced). The secondrow indicates that there were five observations where test T1 was passedwhen B1 was the bad component.

Based on the number of observations of failed tests and bad components,the conditional probability tables are created using probability theory.This is often called the training phase. Once trained, CPTs form thebasis for all Bayesian inference. During the inference phase, whenpresented with a set of failed diagnostic tests, diagnostic system isable to assign probabilities of failure to each component—thus providinga diagnosis of components that might have gone bad.

The confidence measure in the diagnosis of a component is computed inone embodiment as follows:

Define,

variance, v _(ij) =p _(ij)(1−p _(ij))/{α(log N _(ij))} and

confidence, c _(ij)=1−v _(ij)

-   -   Where, p_(ij)=prior probability of test T_(j) failing when        component B_(i) is bad    -   N_(ij)=number of observations involving T_(j) and B_(i)    -   α=a constant (derived from experiments) to normalize variance        between 0 and 1. (Typically, α is a number close to 0.25 to        balance the maximum value of p (1−p))

If N₁ is large or if p_(ij) is close to 0 or 1, confidence is high. Inone embodiment, Log(N_(ij)) is used as opposed to as it is moreappropriate for use in the embodiment described above in CPS 300. In thedata above, T2 seems to be closely correlated with B2 and probability(p₂₂) of T2 failing when B2 is bad is 1.0 (for simplicity, assume nopast history), thus variance is 0 and hence confidence level of B2 withrespect to T2 is 1. Whereas confidence for B1 with respect to T2 isrelatively low (p₁₁ is close to 0.5, thus variance is close to 1 (assumeα is 0.25).

Given a set of failed tests T={T₁, T₂, . . . , T_(t)}, a cumulativeconfidence measure is defined as:

$C_{k} = {\frac{1}{t}{\sum\limits_{j = o}^{t}c_{kj}}}$

for each component B_(k),Where t is the number of tests that failed, and c_(kj) is the confidencelevel of B_(k) with respect to test T_(j).

Another variation of the Cumulative confidence is to weight theindividual confidences with the probabilities as follows:

$C_{k} = {\frac{1}{t}{\sum\limits_{j = o}^{t}{p_{kj}c_{kj}}}}$

for each component B_(k)Therefore, while diagnosing component B_(k), the confidence of a testwith stronger diagnosis will dominate.

Cumulative confidence values are reported along with the inferredprobabilities for each component during diagnosis in one embodiment.Cumulative confidence of a particular component represents the amount offaith the Bayesian network has on its own diagnosis. A smaller value ofconfidence would imply that although it came up with a certainprobability of failure for the component, it does not have enoughconfidence in the inference possibly due to insufficient or ineffectivedata.

The measure of confidence will allow users to be aware of the confidencelevels when evaluating (or further processing) the diagnoses made by thesystem. A user may see a low confidence level in a diagnosis, and decideto investigate further, such as by running more tests, or using visualinspection of a component.

In a further embodiment, an alternative way to compute a confidenceinterval of individual diagnosis made by the diagnostic system is used.The confidence interval defines how close the estimated diagnosis(probability) is to the true diagnosis. As the Bayesian Network in thediagnostic system is trained, its confidence intervals are expected toimprove with time. Therefore, the confidence intervals may be used as anindicator of when the diagnostic system is ready to provide plausiblediagnosis. In one embodiment, a predetermined threshold for a confidenceinterval is used as such an indicator. The threshold may be varied forvarious diagnostic systems as a function of the nature of the underlyingapplication.

In one embodiment, confidence intervals of individual diagnosis arecalculated. Given that a set of tests (in TPS), T={t₁, t₂, . . . t_(n)},has failed, it is assumed that the model has diagnosed components,C={c₁, c₂, . . . , c_(m)}, as bad with probabilities {{circumflex over(p)}₁, {circumflex over (p)}₂, . . . , {circumflex over (p)}_(m)}. Fromthe training data, h_(i), a dataset that associates the test set T andcomponent c_(i), may be identified as follows:

$\begin{matrix}{h_{i} = {{\sum\limits_{\forall{t_{j} \in T}}{fail}_{ij}} + {pass}_{ij}}} & (1)\end{matrix}$

Where fail_(ij)=number of observations where test t_(j) failed whenc_(i) was bad and pass_(ij)=number of observations where test t_(j)passed when c_(i) was bad. Using similar analysis as before, theexpected probability of diagnosis and its confidence interval aredefined as follows.

The expected probability of diagnosing component c_(i) as bad when testset T failed can be expressed as:

E(p _(i))={circumflex over (p)} _(i)  (2)

And corresponding variance as:

v _(i) =p _(i)(1−p)/h _(i)  (3)

where p_(i) is the true probability of component c_(i) being bad whentest set T fails.

If size of the data set h_(i) is large enough, E(p_(i)) has a standardnormal distribution (bell curve) with standard normal random variable Zdefined as:

$\begin{matrix}{Z = \frac{{\overset{\Cap}{p}i} - {pi}}{\sqrt{{{pi}( {1 - {pi}} )}/{hi}}}} & (4)\end{matrix}$

FIG. 4 illustrates a z-curve at 400 in which the shaded area in thez-curve above represents the confidence level P(−z_(α/2)<z<z_(α/2))=1−α.For a normal distribution, given a value of Z (say z), the probabilitydensity function P(−z<z<z) is the shaded area under the under the curvebounded by [−z, +z]. If the value of z=1.96, the area under the curve isapproximately 0.95. That is, P(−1.96<Z<1.96)≈0.95, and hence the valuesof Z in [−1.96, 1.96] define the 95% confidence interval. Similarly,since P(−2.576<Z<2.576)≈0.99, the values of Z in [−2.576, 2.576] definethe 99% confidence interval. The % confidence interval is also expressedas 100(1−α) %.

A table 500 in FIG. 5 lists some of the commonly used confidence levelsand their corresponding α and z_(α/2) values. A 100(1−α) % confidenceinterval (shaded area in FIG. 4.) may be defined as:

P(−z _(α/2) <Z<z _(α/2))=1−α.  (5)

Hence, substituting (4) in (5),

$\begin{matrix}{{P( {{- z_{\alpha/2}} < \frac{{\overset{\Cap}{p}i} - {pi}}{\sqrt{{{pi}( {1 - {pi}} )}/{hi}}} < Z_{\alpha/2}} )}:} & (6)\end{matrix}$

Replacing each ‘<’ by ‘=’ in (6) yields a quadratic equation in{circumflex over (p)}_(i) whose solutions are:

$\begin{matrix}{p_{i} = \frac{{\hat{p}}_{i} + {\frac{Z_{\alpha/2}^{2}}{2h_{i}} \pm {Z_{\alpha/2}\sqrt{\frac{{\hat{p}}_{i}( {1 - {\hat{p}}_{i}} )}{h_{i}} + \frac{Z_{\alpha/2}^{2}}{4h_{i}^{2}}}}}}{1 + {( Z_{\alpha/2}^{2} )/h_{i}}}} & (7)\end{matrix}$

The ‘+’ sign goes with the upper endpoint of the interval and the ‘−’sign goes with the lower endpoint. These two values of p_(i) define theconfidence interval of the true accuracy of the model. It is evident inequation (7) that, if certain tests have been observed to fail moreoften when a particular component c_(i) is bad, then h_(i) would be highand hence the confidence interval would be narrower, indicating that thesystem has more confidence in predicting failure of component c_(i).

A block diagram of a computer system that executes programming forimplementing a diagnostic system with confidence measures as describedabove is shown in FIG. 6. FIG. 6 is an overview diagram of a hardwareand operating environment in conjunction with which embodiments of theinvention may be practiced. The description of FIG. 6 is intended toprovide a brief, general description of suitable computer hardware and asuitable computing environment in conjunction with which the inventionmay be implemented. In some embodiments, the diagnostic system may beimplemented with computer-executable instructions, such as programmodules, being executed by a computer, such as a personal computer.Generally, program modules include routines, programs, objects,components, data structures, etc., that perform particular tasks orimplement particular abstract data types.

Moreover, those skilled in the art will appreciate that the inventionmay be practiced with other computer system configurations, includinghand-held devices, multiprocessor systems, microprocessor-based orprogrammable consumer electronics, network PCS, minicomputers, mainframecomputers, and the like. The invention may also be practiced indistributed computer environments where tasks are performed by I/0remote processing devices that are linked through a communicationsnetwork. In a distributed computing environment, program modules may belocated in both local and remote memory storage devices.

In the embodiment shown in FIG. 6, a hardware and operating environmentis provided that is applicable to any of the servers and/or remoteclients shown in the other Figures.

As shown in FIG. 6, one embodiment of the hardware and operatingenvironment includes a general purpose computing device in the form of acomputer 620 (e.g., a personal computer, workstation, or server),including one or more processing units 621, a system memory 622, and asystem bus 623 that operatively couples various system componentsincluding the system memory 622 to the processing unit 621. There may beonly one or there may be more than one processing unit 621, such thatthe processor of computer 620 comprises a single central-processing unit(CPU), or a plurality of processing units, commonly referred to as amultiprocessor or parallel-processor environment. In variousembodiments, computer 620 is a conventional computer, a distributedcomputer, or any other type of computer.

The system bus 623 can be any of several types of bus structuresincluding a memory bus or memory controller, a peripheral bus, and alocal bus using any of a variety of bus architectures. The system memorycan also be referred to as simply the memory, and, in some embodiments,includes read-only memory (ROM) 624 and random-access memory (RAM) 625.A basic input/output system (BIOS) program 626, containing the basicroutines that help to transfer information between elements within thecomputer 620, such as during start-up, may be stored in ROM 624. Thecomputer 620 further includes a hard disk drive 627 for reading from andwriting to a hard disk, not shown, a magnetic disk drive 628 for readingfrom or writing to a removable magnetic disk 629, and an optical diskdrive 630 for reading from or writing to a removable optical disk 631such as a CD ROM or other optical media.

The hard disk drive 627, magnetic disk drive 628, and optical disk drive630 couple with a hard disk drive interface 632, a magnetic disk driveinterface 633, and an optical disk drive interface 634, respectively.The drives and their associated computer-readable media provide nonvolatile storage of computer-readable instructions, data structures,program modules and other data for the computer 620. It should beappreciated by those skilled in the art that any type ofcomputer-readable media which can store data that is accessible by acomputer, such as magnetic cassettes, flash memory cards, digital videodisks, Bernoulli cartridges, random access memories (RAMs), read onlymemories (ROMs), redundant arrays of independent disks (e.g., RAIDstorage devices) and the like, can be used in the exemplary operatingenvironment.

A plurality of program modules can be stored on the hard disk, magneticdisk 629, optical disk 631, ROM 624, or RAM 625, including an operatingsystem 635, one or more application programs 636, other program modules637, and program data 638. A plug in containing one or more modules canbe resident on any one or number of these computer-readable media.

A user may enter commands and information into computer 620 throughinput devices such as a keyboard 640 and pointing device 642. Otherinput devices (not shown) can include a microphone, joystick, game pad,satellite dish, scanner, or the like. These other input devices areoften connected to the processing unit 621 through a serial portinterface 646 that is coupled to the system bus 623, but can beconnected by other interfaces, such as a parallel port, game port, or auniversal serial bus (USB). A monitor 647 or other type of displaydevice can also be connected to the system bus 623 via an interface,such as a video adapter 648. The monitor 640 can display a graphicaluser interface for the user. In addition to the monitor 640, computerstypically include other peripheral output devices (not shown), such asspeakers and printers.

The computer 620 may operate in a networked environment using logicalconnections to one or more remote computers or servers, such as remotecomputer 649. These logical connections are achieved by a communicationdevice coupled to or a part of the computer 620; the invention is notlimited to a particular type of communications device. The remotecomputer 649 can be another computer, a server, a router, a network PC,a client, a peer device or other common network node, and typicallyincludes many or all of the elements described above I/O relative to thecomputer 620, although only a memory storage device 650 has beenillustrated. The logical connections depicted in FIG. 6 include a localarea network (LAN) 651 and/or a wide area network (WAN) 652. Suchnetworking environments are commonplace in office networks,enterprise-wide computer networks, intranets and the internet, which areall types of networks.

When used in a LAN-networking environment, the computer 620 is connectedto the LAN 651 through a network interface or adapter 653, which is onetype of communications device. In some embodiments, when used in aWAN-networking environment, the computer 620 typically includes a modem654 (another type of communications device) or any other type ofcommunications device, e.g., a wireless transceiver, for establishingcommunications over the wide-area network 652, such as the internet. Themodem 654, which may be internal or external, is connected to the systembus 623 via the serial port interface 646. In a networked environment,program modules depicted relative to the computer 620 can be stored inthe remote memory storage device 50 of remote computer, or server 649.It is appreciated that the network connections shown are exemplary andother means of, and communications devices for, establishing acommunications link between the computers may be used including hybridfiber-coax connections, T1-T3 lines, DSL's, OC-3 and/or OC-12, TCP/IP,microwave, wireless application protocol, and any other electronic mediathrough any suitable switches, routers, outlets and power lines, as thesame are known and understood by one of ordinary skill in the art.

The Abstract is provided to comply with 37 C.F.R. §1.72(b) is submittedwith the understanding that it will not be used to limit the scope ormeaning of the claims.

1. A method comprising: training a computer implemented Bayesian baseddiagnostic system; using the diagnostic system to obtain a diagnosiswith an accuracy measure; and generating a confidence indicatorcorresponding to the diagnosis.
 2. The method of claim 2 wherein theconfidence indicator is generated on a per-diagnosis basis.
 3. Themethod of claim 2 and further comprising generating a cumulativeconfidence indicator for multiple per-diagnosis confidence intervals. 4.The method of claim 1 wherein training the computer implemented Bayesianbased diagnostic system includes generating a conditional probabilitytable.
 5. The method of claim 4 and further comprising using theconfidence indicator to determine when the conditional probability tableis complete and the diagnostic system is sufficiently trained.
 6. Themethod of claim 1 wherein the confidence is calculated in accordancewith:confidence, c _(ij)=1−p _(ij)(1−p _(ij))/{α(log N _(ij))} where,p_(ij)=prior probability of test T_(j) failing when component B_(i) isbad N_(ij)=number of observations involving T_(j) and B_(i), and α=aconstant to normalize variance between 0 and
 1. 7. The method of claim 7wherein α is 0.25.
 8. The method of claim 8 and further comprisinggenerating a cumulative confidence indicator for multiple per-diagnosisconfidence indicators, wherein given a set of failed tests T={T₁, T₂, .. . , T_(t)}, the cumulative confidence indicator is defined as:$C_{k} = {\frac{1}{t}{\sum\limits_{j = o}^{t}c_{kj}}}$ for eachcomponent B_(k), where t is the number of tests that failed, and c_(kj)is the confidence level of B_(k) with respect to test T_(j).
 9. Themethod of claim 8 and further comprising generating a cumulativeconfidence indicator for multiple per-diagnosis confidence intervals,wherein individual confidences are weighted with probabilities.
 10. Themethod of claim 1 wherein the confidence indicator comprises aconfidence interval corresponding to the accuracy of the diagnosis. 11.The method of claim 10 wherein the confidence interval is a function ofa standard normal distribution corresponding to the probability of acertain decision.
 12. A system comprising: a computer implementedBayesian diagnostic system, wherein the diagnostic system includes aninferencing engine and a conditional probability table that forms thebasis for Bayesian inferences once the diagnostic system is trained,each inference including a diagnosis and associated probability of thediagnosis; and a confidence generator to receive the inferences, andgenerate a confidence measure for each inference.
 13. The system ofclaim 12 wherein the confidence generator is operable to generate acumulative confidence measure for multiple inferences.
 14. The system ofclaim 12 wherein the confidence generator is operable to determine whenthe conditional probability table is complete and the diagnostic systemis sufficiently trained.
 15. The system of claim 12 wherein theconfidence generator is operable to calculate the confidence measure inaccordance with:confidence, c _(ij)=1−p _(ij)(1−p _(ij))/{α(log N _(ij))} where,p_(ij)=prior probability of test T_(j) failing when component B_(i) isbad N_(ij)=number of observations involving T_(j) and B_(i), and α=aconstant to normalize variance between 0 and
 1. 16. The system of claim12 wherein the confidence generator is operable to generate a cumulativeconfidence measure for multiple inference confidence indicators, whereingiven a set of failed tests T={T₁, T₂, . . . , T_(t)}, the cumulativeconfidence measure is defined as:$C_{k} = {\frac{1}{t}{\sum\limits_{j = o}^{t}c_{kj}}}$ for eachcomponent B_(k), where t is the number of tests that failed, and c_(kj)is the confidence level of B_(k) with respect to test T_(j).
 17. Thesystem of claim 16 wherein the confidence generator is operable togenerate a cumulative confidence indicator for multiple inferenceconfidence measures, wherein individual confidences are weighted withprobabilities.
 18. The system of claim 12 wherein the confidenceindicator comprises a confidence interval corresponding to the accuracyof the diagnosis.
 19. The system of claim 12 wherein the confidenceinterval is a function of a standard normal distribution correspondingto the probability of a certain decision.
 20. A computer readable devicehaving instruction stored thereon to cause a computer to implement amethod comprising: using a trained computer implemented Bayesian baseddiagnostic system to obtain a diagnosis with an accuracy measure;generating a confidence indicator corresponding to the diagnosis; andstoring the confidence indicator on a computer readable device.
 21. Thedevice of claim 20, wherein the method further comprises generating acumulative confidence indicator for multiple per-diagnosis confidenceintervals.
 22. The device of claim 20 wherein training the computerimplemented Bayesian based diagnostic system includes generating aconditional probability table, and using the confidence indicator todetermine when the conditional probability table is complete and thediagnostic system is sufficiently trained.
 23. The device of claim 20wherein the confidence is calculated in accordance with:confidence, c _(ij)=1−p _(ij)(1−p _(ij))/{α(log N _(ij))} where,p_(ij)=prior probability of test T_(j) failing when component B_(i) isbad N_(ij)=number of observations involving T_(j) and B_(i), and α=aconstant to normalize variance between 0 and
 1. 24. The device of claim23 and further comprising generating a cumulative confidence indicatorfor multiple per-diagnosis confidence indicators, wherein given a set offailed tests T={T₁, T₂, . . . , T_(t)}, the cumulative confidenceindicator is defined as:$C_{k} = {\frac{1}{t}{\sum\limits_{j = o}^{t}c_{kj}}}$ for eachcomponent B_(k), where t is the number of tests that failed, and c_(kj)is the confidence level of B_(k) with respect to test T_(j).
 25. Thedevice of claim 23 and further comprising generating a cumulativeconfidence indicator for multiple per-diagnosis confidence intervals,wherein individual confidences are weighted with probabilities.
 26. Thedevice of claim 20 wherein the confidence indicator comprises aconfidence interval corresponding to the accuracy of the diagnosis,wherein the confidence interval is a function of a standard normaldistribution corresponding to the probability of a certain decision.